Medallion Lecture Local Rademacher Complexities and Oracle Inequalities in Risk Minimization
نویسنده
چکیده
Let F be a class of measurable functions f :S 7→ [0,1] defined on a probability space (S,A, P ). Given a sample (X1, . . . ,Xn) of i.i.d. random variables taking values in S with common distribution P , let Pn denote the empirical measure based on (X1, . . . ,Xn). We study an empirical risk minimization problem Pnf →min, f ∈ F . Given a solution f̂n of this problem, the goal is to obtain very general upper bounds on its excess risk
منابع مشابه
Rejoinder: 2004 Ims Medallion Lecture: Local Rademacher Complexities and Oracle Inequalities in Risk Minimization
of the true risk function F ∋ f 7→ Pf. The first quantity of interest is the L2-diameter of this set, D(F ; δ), and the second one is the function φn(F ; δ) that is equal to the expected supremum of empirical process indexed by the differences f − g, f, g ∈F(δ). These two functions are then combined in the expression Ūn(δ; t) that has its roots in Talagrand’s concentration inequalities for empi...
متن کاملDiscussion of “2004 Ims Medallion Lecture: Local Rademacher Complexities and Oracle Inequalities in Risk Minimization” by v. Koltchinskii
Koltchinskii is to be congratulated for developing a unified framework. This elegant framework is general and allows a user to apply it directly instead of deriving bounds in each risk minimization problem. In the past decade, the problem of risk minimization has been extensively studied in function estimation and classification. In function estimation, it has been investigated using the empiri...
متن کاملDiscussion of “2004 Ims Medallion Lecture: Local Rademacher Complexities and Oracle Inequalities in Risk Minimization” by v. Koltchinskii
1. Introduction. This paper unifies and extends important theoretical results on empirical risk minimization and model selection. It makes extensive and efficient use of new probability inequalities for the amount of concentration of the (possibly symmetrized) empirical process around its mean. The results are very subtle and very pleasing indeed, as they show that oracle inequalities exist for...
متن کاملDiscussion of “2004 Ims Medallion Lecture: Local Rademacher Complexities and Oracle Inequalities in Risk Minimization” by v. Koltchinskii
In this magnificent paper, Professor Koltchinskii offers general and powerful performance bounds for empirical risk minimization, a fundamental principle of statistical learning theory. Since the elegant pioneering work of Vapnik and Chervonenkis in the early 1970s, various such bounds have been known that relate the performance of empirical risk minimizers to combinatorial and geometrical feat...
متن کاملDiscussion of “2004 Ims Medallion Lecture: Local Rademacher Complexities and Oracle Inequalities in Risk Minimization” by v. Koltchinskii
where X1, . . . ,Xn are drawn i.i.d. from a probability measure P on X and F is a class of real-valued functions defined on X . The study of bounds on the expectation P f̂ arises in many applied areas, including the analysis of randomized optimization methods involving Monte Carlo estimates of integrals. Motivated by prediction problems that arise in machine learning and nonparametric statistics...
متن کامل